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Section  l.  Background  and  Definitions  As  for  back  as  1821, 
in  th*  Cours  d'Analyz*  of  tha  ficola  Polytachniqua,  Augustin  Cauchy 
published  a  proof  of  the  following  remarkable  result.  If  any  row, 
together  with  its  matching  column,  is  deleted  from  a  real 
symmetric  matrix,  then  the  eigenvalues  of  the  new  matrix  interlace 
the  eigenvalues  of  the  old  one.  In  the  presence  of  more 

information,  much  more  can  be  said  about  the  interlacing  of 
eigenvalues  and  the  relationship  between  the  spacing  and  the 
corresponding  eigenvectors. 

An  example  of  such  results  can  be  found  in  a  1972  paper  by  G. 
H.  Golub  [2]  which  discusses  aspects  of  the  Lanczos  algorithm. 
Golub,  seeking  bounds  for  eigenvalues,  constructs  a  special  rank 
one  update  H  to  an  n  x  n  symmetric,  tridiagonal  matrix  T  . 

n 

Letting  ?k  be  the  leading  principal  (k+1)  x  (Jt+1)  submatrix  of  H, 
Golub  shows  each  interval  determined  by  the  eigenvalues  of  ? 
contains  an  eigenvalue  of  T  .  Our  Theorem  1  extends  this  by 
replacing  H  with  Tn  itself. 

In  order  to  pursue  these  lines  of  investigation,  some 
notational  convention*  will  prove  helpful.  Let  A^  denote  the 
leading  principal  k  x  k  submatrix  of  a  real  symmetric  matrix  A  = 

n 

A.  The  ordered  eigenvalues  of  Ak  are  denoted  by 

x[k)  s  A™  s  ...  s 

For  each  k  we  assume  we  have  k  orthonormal  eigenvectors  zik> 
associated  with  (where  of  course  we  are  using  the  usual  inner 
product  on  Rb)  .  When  necessary,  the  J th  entry  of  zfk>  is  denoted 
by  z|k>(j).  Frequently,  we  shall  be  concerned  only  with  the 
magnitude  of  th*  last  entry  z‘k)  (k)  and,  for  simplicity,  we  shall 
denote  this  by  the  last  letter  of  the  Greek  alphabet 


“is  - 

Denote  by  pfc(x)  ■  det(xl  -  itfc)  the  characteristic  polynomial  of  Ak 
and  1st  p(x)  -  p  (x) . 

ft 

Section  2  Ha  now  consider  properties  of  the  whole  triangular 


By  Cauchy's  Theorem,  we  know  x‘k  S  \(<k_1 1  s  x^j  ,  as  indicated 
by  the  spacing  in  the  table.  The  first  question  likely  to  arise 
after  looking  at  this  table  is: 

Does  [X*21 ,  x‘2>]  contain  an  eigenvalue  of  *4? 

An  affirmative  answer  does  not  follow  from  Cauchy's  theorem,  and, 
in  fact,  the  answer  is  no  in  general.  However,  the  answer  is  yes 
when  A  is  tridiagonal.  Furthermore,  the  open  interval  (x‘2), 
xj")  must  contain  an  eigenvalue  of  A 4  if  it  is  an  unreduced 
tridiagonal  matrix.  (Recall,  a  tridiagonal  matrix 


is  called  unreduced  if  all  the  b('s  are  nonzero.)  What  is  more 
this  little  known  result  can  be  proved  by  elementary  means 
eminently  suitable  for  introductory  courses  in  linear  algebra.  We 
give  brief,  introductory  proofs  now,  because  we  shall  establish  a 
more  general  result  by  other  means  later.  Here  is  the  general 
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statement,  which  is  sinplifisd  by  letting  A*k>  .  and  xik)  »  os. 

0  kM 

Theorem  1  Let  T  be  an  n  x  n  real  symmetric  unreduced 
tridiagonal  matrix  with  Tk  its  leading  principal  k  x  k  submatrix. 
Then  every  interval  (x|k)/  X(<k’)  contains  a  distinct  eigenvalue  of 
T  /  i  *  0  ,  1  f  «  i  >  f  km 

Note  Theorem  1  is  true  even  if  T  is  reduced  provided  that  we 
replace  the  open  intervals  by  closed  intervals.  This  follows  by 
using  just  a  little  care  after  breaking  T  up  into  tridiagonal 
blocks  at  each  zero  b( .  When  n  -  k  +  1  this  is  just  Cauchy's 
Theorem;  we  shall  need  later  the  fact  that  Cauchy's  Theorem  yields 
strict  inequalities  for  unreduced  tridiagonal  matrices. 

Note  Suppose  A  is  a  n  x  n  symmetric  matrix  and  T  is  a 

partial  tridiagonalization  of  A  (obtained  perhaps  by  the  Lanczos 
algorithm) .  Then  each  interval  determined  by  the  eigenvalues  of 
Tk  contains  an  eigenvalue  of  A,  since  if  we  complete  the 
tridiagonalization  we  obtain  a  Tn  similar  to  A. 

Before  proving  Theorem  1  we  consider  the  special  case  n  *  k  + 

2  given  in  Theorem  2,  below.  For  this  case,  the  proof  is  very 

apparent,  it  illustrates  the  general  case,  and  more  detailed 
information  is  obtained  that  not  only  has  other  interesting 
implications  but  also  hints  at  the  results  to  follow  in  the  next 
section.  In  the  statement  of  Theorem  2  we  use  the  convention 
(a,  b)  »  (a)  if  a  ■  b.  Also,  from  before,  ak<2  is  the  lowest 
diagonal  entry  of  Tk<a,  and  it  must  lie  in  some  interval  [x’k>,  X((k|]. 

Theorem  2  Let  T  be  unreduced  and  assume  a  _  lies  in 

n*a  w 

rx(k‘  x<k,i 

(a)  If  X<k>  s  a  s  X<k*l>,  then  each  of  the  k  +  2 

'  l  k*2  1*1 

intervals 
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(~m  X<k*‘S  (Xlk>  x‘k*l,l  ,x<k>  a  1  fx'**11  x'^S 

(-•,  xt  ),  (Aj  ,  x2  (Xt  ,  aM) ,  (XU1  ,  x^t)f 

(\  ,  \  ).  (XkM  ,  -) 

contains  one  of  the  k  +  2  eigenvalues  of  T  . 

(b)  If  X^j1’  4  «k<2  4  X™  then  (a)  holds  when  the  middle 

two  intervals  are  replaced  by  (Xtk),  x‘kM)),  (a  ,  X(k>). 

r  1  '  i  '  i*i  '  '  '  k*a'  i.i' 

These  relationships  are  illustrated  in  Figure  1. 


Figure  1.  A  possible  graph  of  y  ■  p^2(x) 

Proof  of  Theorem  2  Assume  that  afct2  a  X^*11  (and  the  proof 
in  the  case  ao  „  *  X<k*  is  similar.  By  Cauchy's  Interlace 

Theorem,  we  know  p  „  has  zeros  in  the  ntervals  (-«,  X<k*'1>)  and 
(x;;;X),  «).  we  shall  prove  one  of  the  three  remaining  cases  and 
leave  the  others  as  on  exercise. 

Pick  an  arbitrary  j,  la  j  <  i  and  consider  (x‘k>,  \'k’n). 
The  usual  expansion  of  det(xl  -  T  )  along  its  bottom  row  gives 
the  well-known  recurrence  relationship 

(1)  p  fx)  «  (x  -  a  )p  (x)  -  b2  p  (x) 

v  t  *k*2  '  k»2'*k*lv  k*l*k 

We  know  Pk(x'k))  ■  0  and  PkM(x'kj11)  ■  0.  Using  interlacing  and 
the  factorizations 


Pk(X) 

-  (x  -  xJV 

■•(X  -  x‘k’)  (X  -  x'k|)  •  •  •  (x  -  x‘k) 

) 

;♦*<*> 

-  (x  -  x;k’“)  • 

••(X  -  xJkM’)  (X  -  x£”)...(X  - 

X(k*o 
k*l  ' 

sgn 

P  (X(k*n)  - 

(-l)k'J  and  sgn  PkM(xJk>)  - 

Since  X(k)  <  X(k)  a  a„  .  X(kl  -  a„  ,  <  0.  Putting  this 

J  1  k»2  J  k*2  ^ 

altogether  we  have 

•*-*  p^x\T]  m  sgnl°  ”  Ok(Ct,n  "  '(’1)k'J " 

sgn  pM(x|kl)  -  sgn[(Xi,k)  -  a„,2)PkM (xj**’)  -  0]  -  -(-l)k*w  -  (-l)k‘ 


4- 


Thus  ths  polynomial  pM  has  a  zsro  in  (xjk‘,  \‘k|u) .  u 

Bafors  proving  Theorem  1,  thara  ara  two  comments  to  make. 
First,  Theorem  2  shows  that  though  x'k|2>  is  in  tha  interval 
it  will  tend  to  ba  closer  to  whichever  endpoint 
is  further  from  a^2?  how  close  will  ba  discussed  further  in 
Section  3.  Second,  Theorem  2  shows  that  each  interval  (xtk) , 

i 

x‘kJ)  contains  a  unique  eigenvalue  of  except  for  the  interval 
that  contains  a  .  From  this,  if  we  knew  bounds  of  all  the  a  ,  j 

*♦2  V  J 

e  k  +  2,  than  we  would  know  that  intervals  (X<k>,  X(kl)  outside 
those  bounds  still  contained  unique  eigenvalues  of  T . 

Proof  of  Theorem  1.  Let  n  *  k  +  3.  Let  T*  be  the  trailing 
principal  (n-m+l)  x  (n-m+1)  submatrix  of  T,  which  is  given  by 


a  b 

■  ■ 

b  a 


•  *  i 


a  b 

n-l  n-t 

b  a 

n-t  n 


and  let  p*  be  its  characteristic  polynomial.  Then 

T. 


T  - 


a  b 

k*l  k*l 

b  a  b 

k*t  k*2  k*2 


b  a  b 

k*2  k«3  kO 


k*3 


k  ♦  4 


Now  expand  p(x)  ■  det(xl  -  T)  along  the  ( k  +  2)nd  row  to  obtain 
(after  a  little  algebra) 


P(x)  -  Pk!a(x)P1|M(x)  -  bk\p*3(x)pk(x) 

The  proof  now  proceeds  in  a  manner  very  similar  to  the  proof  of 
Theorem  2.  In  each  interval  (x'k!,  x‘kj)  there  is  at  least  one 
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zero  of  p  in  either  (X*k),  x‘k|u)  or  ( ^ #  x‘k|),  depending  on 

the  relative  position  of  (x‘k) ,  X(k>)  to  the  zeros  of  p  *  and 
p  *  .  in  this,  you  must  uss  the  fact  that  the  zeros  of  p  *  and 
Pk*3  interlace  each  other,  also  (again  by  Cauchy's  Interlace 
Theorem) .  The  details  are  left  as  an  exercise.  H 

Section  3  If  all  that  we  wanted  to  show  was  that  each 
closed  interval  [X^k> ,  X*kj]  contains  a  distinct  eigenvalue  of  A, 
then  the  tridiagonal  assumption  in  Theorem  1  is  not  necessary,  as 
we  shall  show  in  Section  4.  However  it  is  necessary  for  our  next 
result,  which  concerns  what  we  call  crowded  interlacing,  i.  e., 
when  the  next  eigenvalue  along  is  very  close  to  one  of  the  eigen¬ 
values  between  which  it  is  interlaced. 


min{  lx‘k’ 


\  1 ' 


-  xi(k'1,l) 


In  the  context  of  the  Lanczos  algorithm  it  is  important  to 
understand  when  an  eigenvalue  of  T  is  almost  on  top  of  an 
eigenvalue  of  T  1  .  This  phenomenon  is  controlled  by  the  numbers 
cj  introduced  earlier.  This  is  surprising  but  not  without 

Ik 

precedent.  In  studies  of  the  inverse  eigenvalue  problem  it  was 
discovered  independently  by  most  researchers  that  an  unreduced 
tridiagonal  Tk  is  completely  determined  (to  within  t  signs)  by  the 
eigenvalues  (X,n, }"  and  the  "weights"  (w 2  )".  There  are  only  2n  - 
1  free  parameters  here,  since  £  u2  *1.  Among  physicists  the 

i  n 

parameters  ,  u[n )  are  called  action-angle  variables. 

The  following  result  implies,  among  other  things,  that 


-  *;■'> 


u2  (x<n)  -  x(n’) 

tn'  n  t 


Consequently  the  smaller  is  <*>jn  the  closer  are  x‘n_1>  and  x|n) . 

Theorem  3  If  Tn  is  an  unreduced  n  x  n  tridiagonal  matrix  and 
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l  a  k  *  n,  than 


Proof  It  is  known  that  the  tridiagonal  Tk  can  be  recon¬ 
structed  from  the  2k  -  1  values  (*'kl  }k,  {Xi<k’l)  )k‘l.  The 
expression  for  u>lk  is  remarkably  simple.  As  before,  let  pk(x)  = 
det  (rl  -  TJ-  be  the  characteristic  polynomial  of  T k.  Then 


-pk-.(\<k,)/p;(x;k>) 


A  proof  of  this  may  be  found  in  [4,  p.  129].  It  remains  only  to 
reorganize  the  expression  above  as  a  product  of  quotients: 
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u 


lk 


k-i  x(k-n  .  x<k, 

TT  1 - — 


1  '  1 

TT 


j-i 

(  k  ) 


x(k)  -  x(k> 

3*1  1 


1 


^(k-1)  k-t  x(k-l)  x<k> 

^ -  JJ  _J - L_l_ 


.  <  k  >  _  .  <k) 

1  3 


k  -  1 

TT 

j  =  i 


x(k)  -  X 

k _ J 


J  •  1 
(  k  -  t  ) 


x(k>  -  x<k) 

3*1  1 


x,kJ  -  x<k> 

k  J 


i  *  1,  k 


Since  is  unreduced,  Cauchy's  Thaorsa  yialds  x‘kl  <  x‘k'n  < 
X^*',  j  -  1,  . k  -  1,  so  each  factor  in  the  above  products  is 
positive  and  less  than  one.  To  obtain  the  second  inequality  in 
the  theorem,  simply  discard  all  factors  in  each  fj  term  except  far 
the  smallest  one. 

In  order  to  obtain  the  first  inequality  in  the  theorem,  the 
products  given  above  must  be  rearranged  as  follows: 


k  -  1 

TT 

3-1 


i  -  l 

TT 

j  - 1 


k-l 

TT 

j- 1 


k  -  l 

TT 

j-i 


.  (k-t)  _  .  (k) 

A3  1 


.(k-t)  .  (k) 

1  "  1 


x(k)  - 

3*1 

x,kl  - 

t 

^  (k-l) 

3 

>• 

**  Jr 

1 

x(k) 

J 

.  (k-l) 

3 

■c  - 

x<k) 

1 

c- 

x(k-u 

3 

x(k>  - 

k 

k-t  x(k-t.  _  x(k, 

TT  J - - 


J  -a 


.  (k>  .  (k) 

3  '  A1 


.  (k)  .  (k> 

l  “  *1 


.(Ml  _  .  (k) 

l  ”  *1 


t  -  2 

TT 

j  - 1 


(k)  _  .  ( k-l) 

At  3 


x(k>  -  x(k’  , 

i  3*1  J 


x,k>  .  x<k, 
k  1 


x<k)  -  x‘k) 

1  1-1 


k-t  x(k-t)  .  x(k> 

TT  _J - L_ 

11  %  (k)  .  (k) 

3-»*l  X}  -  Xv 


.  (k)  .  <k) 

\  ~  l 


k  -  2 

IT 

3-1 


(k)  .  ( k-l) 

Ak  "  AJ 


(k>  _  .  (k) 

Ak  '  3*1  ) 


x(k’  -  x(k> 

k  l 


.  (k)  .  <k*l) 

X  -  X 

k  k-t 


Cauchy's  Theorem  shows  that  every  quotient  in  the  four 
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products  in  parsnthssss  shown  above  exceeds  one.  Deleting  them 
yields  the  first  inequality,  as  claimed.  B 

Next  we  return  to  full  symmetric  matrices. 

Section  4  We  now  extend  the  results  of  Theorem  1  beyond  the 
tridiagonal  case.  The  eigenvalues  of  the  k  x  k  matrix  A*  define 
k  +  1  intervals  (-*,  ] ,  [At  ,  *2  ] ,  ...,  [A^  ,  ®)  ,  and  we 

want  to  know  when  every  principal  supermatrix  of  A  contains  at 
least  one  eigenvalue  in  each  of  these  intervals.  We  do  not  demand 
that  these  intervals  be  distinct.  Recall  our  earlier  convention 
that  A,k'  *  -•  and  A <kl  *  ». 

0  k*l 


Theorem  4  Let  it 

n 


A 


k 

c 


be  a  symmetric  partition  of  A  . 

n 


(k)  (k) 

If  rank(C)  -  1,  then  each  interval  [ A^  ,  A^],  i  *  0,  1 . k, 

contains  at  least  one  eigenvalue  of  i4n. 

Proof  Let  C  »  wt ,  w  «  Rn‘k,  v  «  Rk.  The  case  of  the 
external  intervals  follows  from  Cauchy's  Interlace  Theorem,  so 
pick  an  i  ■  1,  . ..,  k  -  1. 

If  a *k>  -  A((k| ,  find  a  unit  eigenvector  z  of  A k  associated 

with  A‘k)  and  satisfying  vl  z  ■  0.  Now  let  x  *  [  o  ]  and  it  is 

trivial  to  verify  that  x  is  an  eigenvector  of  corresponding  to 

A<k> . 
i 

Suppose  A (kl  *  A(k) .  Let  z  -  z‘k)  cos  *  +  z‘k)  sin  where 

1  lei  1  1M 

1 1>  will  be  determined  later,  let  x  -  [  o  ] '  and  let  7  ~ 

(a-  +  A^j/2.  Then 


(\[k)  -  r)  z[k)  cos  #  +  (a;;;  -  r)  *;;;  sin 


(A  ~  Tl)x  - 

Now  take  any  #  such  that  vt(z|kl  cos  ^  +  z^  sin  it)  -  0. 


wvt(z‘kl  cos  #  +  zjk|  sin  it) 

At) 


For 
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such  a  #, 


n (-»  -  7l)x»2  -  (Xi(k)  -  7 ) 2  cos2#  +  (X^J  -  7) asin2# 

x<k)  -  x(k‘  I2 

1*1 _ 1 

2 

It  is  a  standard  rssult  (set  [3,  Th.  4-5-1])  that  if  iixii  -  1  and 
UAx  -  7xii  »  5,  than  there  is  an  eigenvalue  of  il  in  [r  -  6,  r  +  {]• 

,  (  (u)  (k) ^ 

If  we  apply  this  to  the  above  where  5  -  X^  -  X  /2,  we  easily 
see  that  there  is  an  eigenvalue  of  A  in  [7  -  a,  7  +  a]  =  [x|k>, 
X*j],  and  we  are  done.  m 

In  general  it  can  be  shown  that  if  rank(C)  -  r,  then  each 
union  of  r  abutting  subintervals  defined  by  the  x|k>'s  holds  at 
least  one  eigenvalue  of  jtn.  As  soon  as  r  exceeds  k,  the  result 
becomes  vacuous. 


It  is  worth  pointing  out  that  Theorems  1  and  3  can  both  be 
obtained  as  special  cases  of  Lehmann's  optimal  intervals. 
Lehmann's  results  [3]  were  published  in  the  1960 's  in  German  and 
are  complicated  by  the  use  of  an  additional  parameter.  He  assumes 
that  Ak  and  C  are  known,  but  U  is  not,  and  then  finds,  for  each  £ 
«  R,  the  smallest  interval  centered  at  £  that  contains  exactly  j 
eigenvalues  of  Aa.  The  answer  turns  out  to  be  that  th^  radius  : 
of  the  smallest  interval  is  the  jth  smallest  singular  value  of 


-  (>:■'  *  >, 

expected. 


It  is  not  hard  to  see  that  if  C  has  rank  one  and  £ 


then  <r 


<Ifc  ]  -  K 


-  X  /2  as 


Section  5  An  alternative  approach  to  Lehmann's  work  was 
taken  by  W.  Kahan.  He  proved  the  following  refined  interlace 


theorem.  See  [4.  pp.  194-7]. 
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Assume  A  »  A  has  the  fora 

ft 

H  Cl  0l  ' 

A  «  C  V  Zl 
0  2V 

where  H  in  m  x  m  and  V  is  k  x  k  (and  0  is  the  zero  matrix)  .  In 

applications,  H  and  C  are  known  but  probably  V,  Z,  and  V  are  not. 

Being  ignorant  of  V  we  replace  it  with  a  k  x  k  X  which  we  are  free 

to  choose  and  define  an  auxiliary  matrix  H  -  M(X)  • 

Denote  the  eigenvalues  of  M  by  m  (X)  and  assume  u  *  -  s 

u  • 

•♦k 

Theorem  5  (Kahai.)  Assume  A,  M,  and  X  are  given  as  above 
where  X  is  any  k  x  k  matrix  satisfying 

v  -  X  is  invertible 

Then  for  each  index  j  -  1,  — ,  m,  the  interval  u^] 

contains  a  different  eigenvalue  Af  of  A.  In  addition,  for  each 
index  i  -  1,  . ..,  k,  there  is  a  different  eigenvalue  of  A 

outside  of  the  open  interval  (u  ,  u  ). 

»  > 

The  only  blemish  in  this  result  is  the  unverifiable 
assumption  that  V  -  X  is  invertible.  Our  final  contribution  is  to 
remove  this  hypothesis  by  looking  carefully  at  the  general  case 
when  v  -  X  is  singular. 

Theorem  6  Kahan's  Interlacing  Theorem  (Theorem  5)  remains 
true  if  the  hypothesis  "V  -  X  is  invertible"  is  removed. 

Proof  Let  A,  X,  and  H  be  as  above,  except  assume  that  v  -  X 
is  singular.  Let  W  ■  Hull  Space  (F  -  X),  so  that  H  •  N1  » 
Picking  orthonormal  bases  for  H  and  H1,  ve  can  change  A  and  M  so 

that  F-X«J"°  where  Y  is  invertible.  Thus, 
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V 


r  * 

ii 

V* 

Ti  fo  0  1  _ 

r  * 

ii 

Ki  * 

.  F21 

K22  . 

"  2  [o  Y  J  '  SO  *  " 

7 

21 

*22. 

22 


*22  +  * 


Break  up  Z  -  [  Zt  I  Z2  ]  ,  compatibly.  Than  for  each  c  >  0,  let  v* 
-  Z/'z‘  +  h  Z.Z1  and  obtain 


c 

C 

o 


1  1 


V  V ‘ 

tl  21 


V  V  z 
21  22  2 


H  0 

H  Cl  0  ' 

F-Cl 

Cl  0  zl 

n  1 1  21  O 

C  V  X  ° 

+ 

C  0  Y  Zl 

2  1  22 

2 

0  0  V-V* 

0  z  z  w* 

L  C  J 

L  1  2  C  J 

T  +  U,  T 


Hence  if  we  let  X_  ■ 


Kl-Cl 


21 


21 

f22 


we  can  now  easily  see  that 


V  -  X  is  invertible  so  that  Kahan's  Interlacing  Theorem  applies 

C 


to  A,  H 


and  X _ 


It  we  carefully  let  c 


0 ,  then  Me 


M 


c'  --c  —  -  - -  - -  -  "e 

(although  W£  /->  H)  so  that  the  eigenvalues  of  M£  go  to  the 

eigenvalues  of  M.  Hence  the  conclusion  follows,  since  A  has 

only  a  finite  number  of  eigenvalues.  _ 


Bibliography 

[1]  G.  H.  Golub,  Some  Uses  of  the  Lanczos  Algorithm  in  Numerical 
Linear  Algebra  in  Topics  in  Numerical  Analysis,  ed.  J.  j.  h. 
Miller,  1973,  pp.  173-184. 

[2]  Y.  Ikebe,  T.  Inagake,  and  S.  Miyamoto,  The  Monotonicity 
Theorem,  Cauchy's  Interlace  Theorem,  and  the  Courant-Fischer 


\7j 


Thaoraa,  The  Am.  Math.  Monthly,  94  (1987)  pp.  352-354. 

(3]  M.  J.  Lehnann,  Optimale  Elgemerteinschiessung ,  Num.  Math.,  5 
(1963)  pp.  246-272. 

[4]  B.  N.  Parlatt,  The  Symmetric  Eigenvalue  Problem, 
Prantiea-Hall,  Englawood  Cliffs,  N.J.,  1980. 


Figura  1.  A  possible  graph  of  y  »  Pk<,2(x) 
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